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Abstract 

Objectives Prior research indicates that public assessments of the manner in which the 
police exercise their authority are a key antecedent of judgments about the legitimacy of 
the police. In this study, the importance of context in influencing people’s assessment of 
police wrongdoing is examined. 

Methods A randomized factorial experiment was used to test how respondents perceive 
and evaluate police—citizens interactions along a range of types of situations and 
encounters. 1,361 subjects were surveyed on factors hypothesized to be salient influences 
on how citizens perceive and evaluate citizen interactions with police. Subjects viewed 
videos of actual police—citizen encounters and were asked for their evaluations of these 
observed encounters. Contextual primes were used to focus subjects on particular aspects 
of the context within which the encounter occurs. 

Results Structural equation models revealed that social contextual framing factors, such 
as the climate of police-community relations and the legality of the stop that led to the 
encounter, influence citizen appraisals of police behavior with effects comparable in size to 
and even larger than demographic variables such as education, race, and income. 
Conclusions These results suggest that the understandings and perceptions that people 
bring to a situation are important determinants of their assessment of police fairness. The 
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police can positively influence citizen interpretations of police actions by striving to create 
a climate of positive police-community relationships in cities. 


Keywords Police legitimacy - Procedural justice - Priming - Randomized 
factorial experiment 


Introduction 


A growing body of research focuses on the legitimacy of the police—how it is constituted, 
when it is conferred, and what are the consequences of giving the police legitimacy or 
withdrawing it on the part of interested residents (Skogan and Frydl 2004; Tyler 2006). 
Legitimacy here means more than simply popular support. Rather, it is a deeper and 
complex notion that takes into account not only public support but also public willingness 
to recognize and defer to official authority. Legitimacy is the public belief that there is a 
responsibility and obligation to voluntarily accept and defer to the decisions made by 
authorities (Parsons 1967; Sarat 1977; Beetham 1991). Police need the support and 
cooperation of citizens to be effective in dealing with crime problems and maintaining 
social order in public spaces (Reisig 2010; Tyler 2004). Social contextual issues, such as 
the climate of police-community relations and the lawfulness of prior police actions, 
contribute to public assessments of the legitimacy of police departments and the right- 
fulness or wrongfulness of the immediate actions of particular officers (Skogan and Meares 
2004; Tyler 2006). 

Researchers in cognitive and social psychology suggest pre-existing social contextual 
knowledge can affect individuals’ processing of ambiguous information (Fiske and Taylor 
1991; Balcetis and Lassiter 2010). “Priming” is an implicit memory effect in which 
exposure to a stimulus influences a response to a later stimulus (Erdley and D’ Agostino 
1988; Higgins et al. 1985). Individual interpretation of information often depends on the 
particular knowledge structures (e.g., concepts and schemas) that are active when exposed 
to an ambiguous stimulus. People primed with varying social contextual factors can react 
very differently to an identical stimulus. For police departments, this suggests that social 
contextual factors shape citizen evaluations of police behavior even when people see the 
same event. As such, citizen assessments of the wrongfulness or rightfulness of particular 
police actions may be powerfully dependent on pre-existing knowledge structures that are 
positive or negative towards the police. 

We used a randomized factorial experimental design and structural equation models to 
examine the influence of varying social contextual factors on individuals’ assessments of 
the appropriateness of police interactions with citizens. In the factorial experiment, sub- 
jects viewed three real-life videos of police—citizen interactions presented in counterbal- 
anced order. Prior to viewing the videos, subjects were randomly allocated positive, 
neutral, and negative descriptions for four social contextual issues: the climate of police— 
community relations in the city where the interaction takes place, the lawfulness of the 
police stop, the officers’ prior history, and the citizens’ prior history. Our analyses revealed 
that, although participants viewed identical videos, variations in the social contextual 
framing of the police—citizen interactions shown produced large and significant effects on 
subjects’ ratings of police behavior in the video. 
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This article begins by briefly reviewing the literature on police legitimacy and the 
salience of social contextual factors in individual appraisals of police behavior in inter- 
actions with citizens. The research design, data, and analytical framework are presented in 
subsequent sections. Results of the randomized factorial experiment and supporting 
structural equation model analyses are then detailed. The concluding section summarizes 
the research findings, frames the results within the existing literature on police legitimacy, 
and presents the implications of the research for police policy development and practice. 


Literature Review 
Police Legitimacy and Police Behavior 


Policing communities, especially minority communities, always involves a delicate bal- 
ance (Meares and Kahan 1998). On the one hand, research suggests that the police benefit 
from the general willingness of community members to cooperate with them to report 
crimes, identify criminals, assist in investigations, and address conditions that might 
facilitate crime (Moore 1992; Reisig 2010; Tyler and Fagan 2008). On the other hand, 
effective policing invariably involves tactics that bring the police into close and regular 
contact with community residents. This contact can be viewed by community residents, 
particularly minority residents, as intrusive and unwarranted, leading citizens to doubt 
whether the police respect their rights and care about their well being (Carr et al. 2007; 
Brunson and Miller 2006; Brunson and Weitzer 2009). Whether or not individuals have 
personal contact with police officers, their perceptions of the legitimacy of police have 
important consequences for police effectiveness (Tyler 2004, 2006). Policing is far more 
difficult without the support of the public. Therefore, police effectiveness is powerfully 
influenced by the consequences of different tactical and policy choices for their legitimacy. 

Legitimacy is regarded as a view among the members of the communities involved that 
legal authorities play an appropriate role in making and implementing rules governing 
public conduct (Sunshine and Tyler 2003; Tyler and Huo 2002). Obviously, legal 
authorities can shape public behavior by virtue of their ability to use force and commu- 
nicate the risk of sanctioning for noncompliance. However, in democratic societies the 
authorities rely heavily upon public willingness to voluntarily defer because they view the 
conduct of the police and courts as reasonable and just (Scheingold 1974; Easton 1975). 
These ethical qualities distinguish legitimacy from coercion, and reflect the idea that 
authorities have trust and confidence among the public, i.e. the view that people consider 
agents of social control to be acting on behalf of the community and exercising their 
authority in accord with principles of reasonable and appropriate conduct (Parsons 1967; 
Sarat 1977; Beetham 1991). Such feelings create legitimacy—the view that authorities 
ought to be deferred to and obeyed. It is such legitimacy, in turn, that leads to public 
cooperation with the police, cooperation that is central to their ability to manage crime and 
social disorder within communities (Tyler and Wakslak 2004). 

It is obviously important to understand the factors that shape police legitimacy. One 
central focus of past research has been upon the strategies used by the police to manage 
crime (see, e.g. Braga 2008; Weisburd and Eck 2004). However, studies consistently 
suggest that public views about the police and the law are not simple reflections of the 
crime rate (Moore 2002; Tyler 2004). Recent discussions of policing acknowledge the 
important role that policing strategies have in shaping crime, but argue that attention needs 
to be given to a larger framework within which the influence of police actions on police 
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legitimacy in the eyes of the public is also studied (Skogan and Frydl 2004). Such concerns 
have fueled a series of reforms in the manner in which legal authorities act, including 
community policing and neighborhood courts (Fagan and Malkin 2003). 

Research suggests that the public evaluates police not only in terms of the ability of the 
police to fight crime (see, e.g. Moore 2002). People are also sensitive to the manner 
through which the police exercise their authority. As a consequence, the procedural justice 
of police actions is central to police legitimacy and policies such as racial profiling that are 
not evaluated to be just undermine police legitimacy (Fagan 2002; Tyler and Wakslak 
2004). The issue of police behavior is very important in shaping the views of the public. 
Studies in the United States emphasize that both Whites and minority group members 
focus on the manner in which the police exercise their authority both when making general 
evaluations of the legitimacy of the police (e.g. Sunshine and Tyler 2003) and when 
reacting to personal experiences with the police (e.g. Tyler and Huo 2002). 

Studies of personal encounters with the police consistently document that post-expe- 
rience feelings are determined by the fairness in which the problem was handled. The 
National Research Council’s Committee to Review Police Policy and Practices identifies 
four dimensions of fairness in police—citizen interactions (Skogan and Frydl 2004). First, 
citizens need to have meaningful participation in interactions. Importantly, citizens must 
have the ability to explain situations and communicate with the police. Second, citizens 
need to feel that the police officers were neutral in their assessments of situations by using 
objective indicators to make decisions rather than personal views. Third, citizens must feel 
that they were treated with respect and dignity by the police during interactions. Fourth, 
police officers need to inspire trust in the citizenry. If people believe authorities care about 
their well-being and are considerate of their needs and concerns, they view procedures as 
fairer. Police can encourage the public to view them as trustworthy by explaining their 
decisions and accounting for their conduct. 

Tyler (2003) proposed a process-based model of police legitimacy that suggests a direct 
and measurable relationship between how police treat citizens and then, in turn, what 
citizens think of the police. Two related components are used to evaluate whether police 
practices are procedurally-just in the eyes of the public: quality of decision making (e.g., 
officers’ use of objective indicators to make decisions) and quality of treatment (e.g., 
authorities treat citizens with dignity and respect) (Tyler 2003; Reisig et al. 2007). If the 
police are perceived to make fair decisions and to treat people respectfully, they will be 
viewed as legitimate authorities. As a result, the police will enjoy enhanced citizen 
cooperation and compliance with the law. A series of research studies have been found to 
support the key theoretical constructs in the process-based model of police legitimacy (see, 
e.g. Mastrofski et al. 1996; McCluskey 2003; Paternoster et al. 1997; Sunshine and Tyler 
2003; Tyler and Fagan 2008; Tyler and Wakslak 2004). 

Research has shown that other factors may also be quite important in stimulating citizen 
cooperation with the police. For instance, using survey data from the United States, Reisig 
et al. (2007) found that the fair distribution of police resources across individuals and 
neighborhoods (i.e., distributive fairness) was correlated with cooperation with the police 
and, using survey data from Ghana, Tankebe (2009) showed that perceived police crime 
control effectiveness was the main factor that determined citizen cooperation. Jackson and 
Bradford (2009) find that people think about their local police in ways less to do with the 
risk of victimization and more to do with judgments of social cohesion and moral con- 
sensus. Further, Jackson et al. (2012) found that people accept the police’s right to dictate 
appropriate behavior not only when they feel a duty to obey officers, but also when they 
believe that the institution acts according to a shared moral purpose with citizens. Drawing 
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on the work of Beetham (1991) and Coicaud (2002), Bottoms and Tankebe (2012) suggest 
effectiveness, distributive fairness, procedural fairness, and lawfulness as some of the 
likely main contents of the dimensions of police legitimacy in liberal democracies. 
Through an analysis of London survey data, Tankebe (2013) found empirical support for 
the relevance of these four dimensions to police legitimacy and that legitimacy has both a 
direct and an indirect influence on people’s willingness to cooperate with the police. 

Pre-existing opinions of the police strongly shape citizen perceptions of their interac- 
tions with the police (Brandl et al. 1994; Rosenbaum et al. 2005). People are suggested to 
form general impressions of the police before they have any personal contact with them; 
this, in turn, influences the nature of interactions between individuals and the police when 
such contact occurs (Hawdon 2008). Moreover, vicarious experiences, such as stories that 
people hear from their friends, family, and the media, influence the way citizens interpret 
and evaluate their own encounters with the police (Brunson 2007; Gallagher et al. 2001; 
Hohl et al. 2010; Weitzer and Tuch 2006). For instance, Warren (2011) found that people 
who hear negative stories about the police from family and friends are four times more 
likely to perceive disrespect during their own encounters with the police. In the next 
section, we more closely examine the potential importance of prior attitudes and expec- 
tations held by people in assessing police behavior in police—citizen encounters. 


The Salience of Social Contextual Factors in Assessing Police Behavior 


Many decades of psychological research has demonstrated that perceptions of events are 
shaped by the attitudes and expectations that people bring into the situation in which those 
events occur. So, different people can look at the same objective event and subjectively 
experience that event differently. People often remember the “facts” differently, as well as 
forming different inferences about the character and the intentions of those involved. In a 
classic demonstration of this phenomenon, Hastorf and Cantril (1954) interviewed psy- 
chology students from Dartmouth and Princeton after they observed the same football 
game. They were asked to evaluate what had happened and how fairly the players on each 
side played. Despite the fact the two groups had seen the same football game, there were 
large differences in both their evaluations of what had happened and their views about how 
fairly each side had played. Subsequent studies reinforce the suggestion that people’s prior 
attitudes and values shape their understanding of the “facts” and their evaluations of and 
reactions to social situations (e.g., see Fiske and Taylor 1991). 

Of course, perceptions may be simultaneously influenced by people’s own ex ante 
attitudes and values; those factors may then interact with the characteristics of the social 
situations in which they are involved (Balcetis and Lassiter 2010). A particularly dramatic 
example of this phenomenon in a legal setting is provided in recent research on racial 
profiling (Tyler and Wakslak 2004). One question that has been studied is when people 
perceive themselves to be the victims of profiling. Research suggests that when they are 
treated disrespectfully by police officers during a street stop, minority group members are 
more likely to infer that they were stopped because of their race (Tyler and Wakslak 2004). 
And once they have made this inference about the police officer involved, people are likely 
to “remember” the facts of their experience in a manner consistent with this inference. In 
one study, analysis of videotapes of police—citizen encounters showed that motorists who 
had been stopped and had complained of being profiled remembered “facts,” such as being 
called racially-tinged names (Sherman 1999). However, coding of the videotapes sug- 
gested that officers were interpersonally disrespectful but not overtly racist as the com- 
plaining motorists remembered. 
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Psychologists have developed a variety of procedures for studying the influence of prior 
situational context upon the reactions of people with different attitudes and values to social 
situations. One of the most influential is priming (Bargh and Chartrand 2000). Priming 
involves the activation of particular aspects of a person’s prior mental framework by 
presenting them with a brief set of stimuli designed to direct their thoughts and feelings in 
a particular manner. This presentation can evoke either conscious or automatic thought 
processes, but the key point is that it leads people to focus upon related thoughts. So, for 
example, by priming people with concepts such as adventurous or independent, we can 
lead them to form more positive evaluations of a person engaged in activities such as 
sailing across the world (Higgins et al. 1977). When police and probation officers were 
primed with racialized cues about hair, clothing and music styles (e.g., “dreadlocks, baggy 
jeans, hip hop”), the group that was “racially” primed attributed greater culpability to a 
racially ambiguous offender in a vignette about a convenience store robbery (Graham and 
Lowery 2007). 

These “priors” also affect how people understand the “facts” in a given setting. As an 
example, eyewitness identification research makes clear that people’s evaluations of others 
are shaped not only by the objective events they have witnessed, but also by their prior 
attitudes and values (Wells and Olson 2003). Further, studies suggest that the context 
within which people make evaluations shapes those evaluations (Fiske and Taylor 1991). 
When people are threatened by fear of crime, or have prior negative associations with the 
members of particular groups, these aspects of their mental frameworks shape their 
understanding of and reactions to events. 

The procedural justice literature provides a related set of findings that suggests situa- 
tional cues and framing effects may be important in affecting a person’s evaluation of a 
situation. Previous research on a variety of law enforcement issues establishes that legal 
institutions gain and preserve legitimacy through adherence to procedural fairness norms 
(Tyler 2003, 2005). That is, judgments about the fairness of the manner in which these 
actors exercise their authority influence evaluations of legitimacy of the police and other 
legal actors in critically important ways. Such procedural justice judgments are found to 
both shape reactions to personal experiences with legal authorities and to be important in 
assessments based upon the general activities of the police (e.g., Paternoster et al. 1997; 
Reisig and Chandek 2001). In both instances, citizens view the police and courts as less 
legitimate when they personally experience or vicariously become aware of instances of 
unfair, disrespectful or unethical treatment—in other words, procedural injustice (Tyler 
2004). Accordingly, experiences with the law and legal actors will shape attributions of the 
legitimacy of those actors and the institutions they represent. 

There are many examples that illustrate this dynamic. For example, respectful and polite 
treatment by police reduces perceptions of racial profiling (Tyler and Wakslak 2004), and 
such treatment combined with officer efforts to explain the reasons for their actions 
translates into feelings of citizen satisfaction with police more generally (Skogan and 
Hartnett 1997; Tyler and Huo 2002). These legitimacy gains, in turn, translate into higher 
levels of respect for law and cooperation with law enforcement efforts. A large social 
science literature now examines these relationships and corroborates these findings in 
numerous distinct contexts (Tyler 2003; Tyler et al. 1997). Conversely, the lack of just 
legal and political procedures is found to motivate collective actions, such as riots, and to 
encourage people to become committed to crimes and even to extreme acts of terrorism 
(Krueger and Malechkova 2003; McCauley 2002). 

Documenting these kinds of framing effects makes an important contribution to the 
existing literature on policing. The authors of a recent literature review on police—citizen 
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relations conclude “there is no consensus as to which combinations of variables explain the 
greatest variance in attitudes toward the police” (Brown and Benedict 2002, p. 564). While 
the existing literature has explored individual-level demographic predictors, much less 
attention has been given to non-demographic factors such as the public’s personal expe- 
riences and what people learn from the media and from friends (Gallagher et al. 2001). 
Similarly, the research regarding interactions among neighborhood or city conditions and 
evaluations of police is slim (Weitzer and Tuch 2006). We address some of these voids and 
add an important new dimension: situation-specific characteristics of encounters. Specif- 
ically, we assess the importance of situation-specific perceptions that are held just prior to 
viewing a situation in which police make an arrest. As we discuss in detail below, what we 
find is that differences in these immediate prior perceptions have effects that are as big as 
or bigger than demographic variables such as race, education, and income that have been 
the focus of prior research. 


Research Design and Data 


We used a randomized factorial experimental design to determine whether differences in 
social contextual framing factors produce variations in attributions of legitimacy to police 
during police—citizen encounters. A factorial experiment is a research design that consists 
of two or more factors that have discrete possible values or levels where the experimental 
units of analysis are distributed across all possible combinations of these levels (see Fisher 
1926; Box et al. 2005). Factorial experimental designs allow the analysis of the effects of 
each factor on the outcome variable as well as interactions between these factors and the 
outcome variable. Subjects were first given a description of the context in which an arrest 
took place. The experimental manipulation randomly varied four social contextual factors: 
the description of the climate of police-community relations in the city in which the 
incident occurred; a description of the legal context of the police stop shown in the video, a 
description of the police officer involved; and a description of the citizen involved. After 
having been given a description of the context, subjects were then shown a set of three 
videos of actual police—citizen encounters and asked to evaluate the appropriateness of the 
observed police behavior.' The goal of the experiment was to assess the degree to which 
peoples’ judgments about police behavior can be influenced by how an incident is framed. 

As will be described further below, subjects were exposed to the varying social con- 
textual factors through a written script; they then viewed a video and responded to a battery 
of questions to measure the impact of these factors on their assessment of the police 
behavior viewed. The randomization of the social contextual factors mentioned in the 
written scripts was intended to ensure that the subjects in each of the factorial groups were 
equivalent. In general, when randomization is successfully achieved and the subjects in 
varying groups are equivalent, the estimation of treatment net effects can be simply 
expressed as the experimental group’s score on an outcome measure minus the control 
group’s outcome score (plus or minus stochastic effects; see Rossi et al. 2004). As part of 
this design, we collected additional data on a variety of dispositional variables and prior 
experiences for each subject to determine whether, the randomization of subjects among 


' Prior studies suggest the value of this type of vignette approach (Seron et al. 2006), but have used written 
vignettes of uncertain realism. We utilized actual videos of police interactions centered on the core issues in 
the study to supplement the vignettes and provide salient visual cues and contextual primes to stimulate 
respondents. 
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the varying social contextual factors created equivalent factorial groups. These analyses, 
described below, confirmed that randomization was achieved and, reasonably equivalent 
groups were created. 

It is important to note here that there are very few randomized controlled trials in this 
important area of policing research. A recent Campbell Collaboration systematic review 
of legitimacy policing concluded, “there is a clear lack of randomized experiments in the 
international research literature that specifically seek to isolate and test the component 
parts of a legitimacy policing intervention” (Mazerolle et al. 2012, p. 10). One note- 
worthy exception is the Queensland Community Engagement Trial which examined the 
direct and indirect outcomes of procedural justice policing under randomized field trial 
conditions. Relative to routine traffic stops in the comparison group, Mazerolle et al. 
(2013) found procedurally-just traffic encounters in the treatment group significantly 
shaped citizen perceptions of the actual encounter directly and general views of the police 
indirectly. 

Randomized experimental designs have the advantage of strong internal validity—the 
extent to which a research design can eliminate competing explanations of an observed 
correlation (Shadish et al. 2002). However, randomized experiments are also noted to 
suffer from low external validity—the extent to which the findings of a study can be 
generalized to the population of interest. External validity of randomized experiments 
can be improved by drawing an initial sample that is representative of the population 
(Rossi et al. 2004). In this study, we were interested in making generalizations to urban 
populations. The study sample was constructed through a contractual arrangement with 
Knowledge Networks (hereafter, KN), a survey research firm based in Menlo Park, 
California that specializes in video-based surveys using the internet. KN retains a per- 
manent sample of approximately 40,000 plus individuals from which subsamples can be 
drawn to carry out specific surveys. The KN panel sample is representative of the U.S. 
population, which allows for stratification and representation across demographic groups 
(Couper 2000; Dennis 2001). Recruitment into the KN panel takes place through a 
Random Digit Dialing (RDD) method to random samples of U.S. households. All 
members of the household are invited to participate in the panel. Surveys are answered 
electronically either over the PC or through a web-based TV connection. KN panel 
ane nes receive free internet access and email accounts, including equipment if 
needed. 


2 KN has completed several studies to validate its methodology, including both accuracy and nonresponse 
bias. For example, Krosnick and Chang (2001) compared responses to a single survey on the 2000 presi- 
dential election using three different samples and methods: KN, Harris Interactive (HI), and a RDD survey 
conducted by the Ohio State University Center for Survey Research. They reported that the KN samples had 
higher completion rates, closer approximation to census estimates, and less skew on basic measures of 
opinion and voting behavior. They also show higher measurement reliability and “non-differentiation” of 
responses for Internet-enabled data collection compared to RDD. Skitka and Mullen (2002) and Skitka et al. 
(2002) show that on questions of civic values and actions, in particular, telephone survey responses were 
more often influenced by respondents’ motivations to present themselves in a positive light to interviewers 
compared to internet-enabled surveys. Benchmarking studies show that KN samples yield estimates for 
several health behaviors—current smoking, diabetes, ulcer, migraine headaches, and stroke—similar to the 
estimates in the National Health Interview Survey conducted by telephone annually by CDC. The average 
difference in the results is about 1 percentage point (Baker et al. 2003). 
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Data 


The study sample included KN panel members age 18 or older who lived in 15 U.S. cities.* 
A total of 1,361 completed interviews were obtained, reflecting a 62 % completion rate.* 
Respondents viewed three videos of citizen—police encounters presented in random order. 
After each video, the respondents completed a set of questions evaluating whether they 
believed that the police officer(s) in the video were wrong in their handling of the situation 
and deserved to be punished. It was critical to determine whether the effects of group 
membership, past experience with the police and crime, and general perceptions of the 
police, and other salient factors were significantly different among subjects in the groups 
randomly exposed to the varying social contextual framing effects. As such, each 
respondent also completed a questionnaire on their identification with the police, political 
ideology, trust in government, perceptions of police effectiveness in their neighborhood, 
perceptions of procedural justice in their most recent encounter with the police, prior crime 
victimization, disorder concerns in their neighborhood, and fear of crime in their neigh- 
borhood.° This also allowed us to test whether the above factors continue to affect their 
evaluation of police behavior under experimental conditions. 

Table 1 presents selected socio-demographic characteristics of the respondents. The 
KN panel members that participated in the study were mostly older than 30 years of 
age (78 %). Non-Hispanic whites (59 %) represented the largest racial group of 
respondents followed by Hispanics (18 %) and non-Hispanic blacks (14 %). The sample 
was evenly split between males and females. Slightly more than 34 % of respondents 
had at least a bachelor’s degree and only about 9 % had less than a high school 
education. The sample was generally middle-class and higher with 33 % having a 
yearly household income between $40,000 and $75,000 and nearly 41 % having a 
yearly household income greater than $75,000. Respondents were politically diverse as 
36 % described themselves as Independents, 34 % described themselves as Democrats, 
and 27 % described themselves as Republicans. Half of the respondents reported that a 
member of their household had been the prior victim of a property or violent crime. It 
is unfortunate that we were not able to obtain additional young, minority subjects. 
While our final sample was not completely representative of an urban population, we 
did obtain a diverse group of respondents that strengthened the external validity of our 
randomized factorial experiment. 


3 The 15 U.S. cities were Baltimore, Boston, Chicago, Cleveland, Denver, Detroit, Houston, Los Angeles, 
Miami, New York, Philadelphia, Phoenix, San Diego, Seattle, and Washington, DC. Knowledge Networks 
completed between 85 (New York) and 98 (Baltimore) interviews in each city. 


4 The survey was fielded on March 27, 2008 to a sample of 2,183 Knowledge Networks panel members. To 
be included in this analysis, a respondent had to view all three videos (N = 1,361). 


> These questions were randomly administered before or after the videos were viewed. 676 respondents 
completed the questionnaire after seeing the videos and 685 respondents completed the questionnaire before 
seeing the videos. To investigate whether the timing of the administration of the questionnaire impacted our 
findings, we split the sample and replicated the full sample analyses presented here. When the impact of the 
four framing effects dummy variables (city climate of police-citizen relations, stop context, officer history, 
and citizen history) on the “police were wrong/should be punished” outcome variable were estimated, the 
two split samples produced coefficients of similar direction, magnitude, and statistical significance 
(p < .05). As such, whether the respondent answered the questionnaire before or after the video did not 
substantively impact the findings presented in Tables 2, 3, and 4. 


va Springer 


J Quant Criminol 


Table 1 Selected socio-demographic characteristics of respondents 


Variable Number Descriptive statistics 

Age 1,361 Mean = 46 years, range = 1-88 years 
Less than 30 years old = 22.2 % 

Sex 1,361 47.7 % male, 52.3 % female 

Race 1,361 White, non-Hispanic = 58.6 % 


Black, non-Hispanic = 14.3 % 
Hispanic = 18.0 % 
Other = 9.1 % 
Education 1,361 Bachelor’s degree or higher = 34.2 % 
Some college = 26.7 % 
High school = 30.6 % 
Less than high school = 8.5 % 
Household income 1,361 Greater than $75,000 = 40.9 % 
>$40,000 and < $75,000 = 32.7 % 
Less than $40,000 = 26.4 % 
Political identification 1,354 Independent = 36.0 % 
Democrat = 34.2 % 
Republican = 27.1 % 
Other = 2.7 % 
Conservative scale 
Range = 0 ‘weak’—100 ‘very strong’ 
Mean = 47.4 
Prior crime victim 1,361 Household never victimized = 50.1 % 


Household victimized at least once = 49.9 % 


Prior Dispositions of Respondents 


The questionnaire included several sets of questions to measure variables of interest that 
could not be directly observed, known as “latent variables” (Long 1983). These vari- 
ables were included as controls in the analysis below. Seven latent variables were 
constructed by analyzing the covariance of the responses collected from these sets of 
questions. The questions were designed to capture prior dispositions of respondents and 
included items that measured the constructs presented below. Unless otherwise noted, the 
observed variables were four-point Likert scales (for example: agree strongly; agree; 
disagree; disagree strongly). We used the Cronbach’s Alpha statistic to measure the 
internal consistency of the observed variables that comprised the latent variables 
(Cronbach 1951). Alpha varies from zero to one with higher values indicating a greater 
degree of reliability in measurement. As will be shown below, all the latent variables had 
Cronbach’s Alpha statistics that equaled or exceeded .75, suggesting good internal 
consistency across the observed variables that comprised the latent variables. The con- 
firmatory factor analysis loadings for the endogenous variables that comprise our latent 
variables are provided in the results section. All endogenous variables loaded very highly 
on their respective factors. 
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Identification with the police 


Psychological research has shown that individuals who identify with and feel connected to 
particular groups are more likely to view members of this group more positively and 
members of other groups more negatively (e.g. Gaertner et al. 1989). Five questions on a 
respondent’s identification with police were included (alpha = .85). Respondents were 
asked to agree-disagree that: “If you talked to most of the police officers who work in your 
neighborhood, you think you would find they have similar views to your own on many 
issues”; “You can usually understand why the police who work in your neighborhood are 
acting as they are in a particular situation”; “You generally like the police officers who 
work in your neighborhood”; “If most of the police officers who work in your neigh- 
borhood knew you they would respect your values”; and “Most of the police officers who 
work in your neighborhood would value what you contribute to your neighborhood”. 


General Political Ideology 


Relative to those with liberal political views, individuals with conservative views are more 
likely to hold positive opinions of law enforcement and the police (Tyler et al. 1997). Four 
questions on respondents’ general political ideology were included (alpha = .82): “Where 
would you place yourself on a scale of political views from extremely liberal to extremely 
conservative?”; “Generally speaking, do you think of yourself as a strong democrat to a 
strong republican?”; and “Can you tell me your feelings about liberals and conservatives 
using a thermometer scale ranging from 0 to 100 reflecting cold to hot (50 is no feelings)?” 


Trust in Government Intervention 


Research has found that individuals who trust in the U.S. government were more likely to 
obey laws and show willingness to defer to authorities (Tyler and Mitchell 1994; Tyler and 
Huo 2002). Three questions measuring an individual’s confidence in U.S. government 
intervention were included (alpha = .83). Respondents were asked to agree—disagree that: 
“Some people think that the government in Washington should do everything possible to 
improve the standard of living for all Americans.”; “Some people think that the govern- 
ment in Washington is trying to do too many things that should be left to individuals and 
private businesses”; and “Some people think that racial minorities in the US have been 
discriminated against for so long that the government has a special obligation to help 
improve their living standards.” 


Evaluations of Police Effectiveness 


Citizens who view the police as doing a poor job in dealing with crime express less 
satisfaction with the police and lower confidence in the police (Weitzer and Tuch 2006). 
Three questions on police effectiveness were included (alpha = .84). Respondents were 
asked: “How effective are the police in fighting crime in your neighborhood?” (measured 
“very effective” to “not effective at all”); “When people in your neighborhood call the 
police to ask for help how quickly do the police respond” (measured “very quickly” to “no 
response at all”); and “How effective are the police in your neighborhood at helping 
people who ask them for help?” (measured “very effective” to “not effective at all’). 
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Fear of Crime 


Feelings of personal safety influence citizens’ approval of the police (Weitzer and Tuch 
2006). When citizens are fearful o being a crime victim, their satisfaction with the police 
decreases. Three questions on respondents’ fear of crime were included (alpha = .75). 
Respondents were asked: “How worried are you about being a crime victim?”; “How 
worried are you about your home being broken into?”; and “How worried are you about 
being robbed, assaulted or mugged in your neighborhood?” For these three questions, 
response items included “a great deal,” “somewhat,” “a little,” and “not much at all.” 


Concerns About Neighborhood Disorder 


Fear of crime has been found to be correlated with social and physical disorder in neigh- 
borhoods (Skogan 1990). Five questions on respondents’ concerns with disorder in their 
neighborhoods were included (alpha = .87). Respondents were asked: “How often do you 
see these conditions in your neighborhood?”: “Empty beer bottles on the streets or side- 
walks”; “Graffiti on the walls”; “Gangs hanging out on the streets”; “People drinking beer, 
wine or liquor on the street” ; and “People buying or selling drugs on the street” . For these five 
questions, response items included “usually,” “sometimes,” “rarely,” and “almost never.” 


9 6 


Personal Experience with the Police 


Police treatment of citizens during an encounter affects citizens’ views of the police more 
strongly than the outcome of the encounter (i.e. problem resolution, citation, or an arrest). 
When police treat citizens unfairly, brusquely, or discourteously, the experience has a 
negative influence not only on their assessments of the immediate encounter but also of their 
overall opinions of the police (Tyler 2006; Reisig and Chandek 2001). Some 66.8 percent of 
the 1,361 respondents reported a prior encounter with the police involving a crime or some 
possible violation of the law such as a driving infraction. For those who had at least one 
previous encounter with the police, the following six procedural justice questions were asked 
on their most recent interaction (alpha = .93): “Thad the opportunity to describe my situation 
before decisions were made”; “I was treated the same way that others would be treated in a 
similar situation”; “The police made their decisions based upon facts”; “The police cared 
about my concerns”; “The police treated me politely”; and “the police respected my rights”. 


Manipulated Social Contextual Variables 


Citizen experiences with police and other domains of criminal law take place within a larger 
social and political context of police-community relations. Incidents such as the Rodney 
King beating in Los Angeles, or the shooting of Amadou Diallo or Sean Bell in New York, 
or riots in Cincinnati against perceived police abuses may have framing effects on the ways 
that citizens perceive, interpret and evaluate police actions. Information on problematic past 
behavior of individuals involved in police actions, such as the past use of racial epithets by 
Detective Mark Fuhrman in the O.J. Simpson murder trial or whether the citizen in the 
interaction is a previously convicted felon, may similarly influence how citizens appraise 
police behavior. Knowledge of whether the initial stop was lawfully conducted by officers 
may also impact how citizens assess subsequent police behaviors during the same incident. 
Accordingly, we also suspect that these framing events influence not only how citizens 
perceive and evaluate police—citizen interactions, but they also moderate the salience of 
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individual and general experiences of citizens in their influence on the evaluation of the 
legitimacy of police actions. We hypothesized that alternate descriptions of social contextual 
factors would influence our respondents’ appraisals of police behavior. 

Prior to viewing a video, respondents were given background information about the social 
context of that video. That information addressed four issues: the climate of police-community 
relations in the city, context of the stop, officer history, and citizen history. In each contextual 
variation, there were three framing conditions: negative, neutral, and positive. Respondents 
were randomly allocated these three framing conditions for each of the four issues resulting in 
81 different possible combinations (3* = 81 groups of subjects in the factorial experiment). 


City Climate of Police-Community Relations 


Respondents were told that the police department had been cited for its strong community 
policing work, had received extensive negative media coverage for poor community 
relations and civil rights violations, or no information was provided. 


Stop Context 


The stop context was described as legal, characterized as illegal, or neutral information was 
provided. 


Officer History 


The officer(s) in the videos had either been recommended for a commendation recognizing 
high quality and integrity in their work, had been recommended for in-service training to 
address inconsistent decision making in enforcing the law, or no information was provided. 


Citizen History 


The citizen(s) in the videos were presented as having a prior criminal record, not having a 
prior criminal record, or no information was provided. 

Polychotomous dummy variables were constructed to represent the framing conditions for 
each social context issue. Using “police history” as an example, we created two dummy 
variables to represent negative (0 = not negative, 1 = negative) and neutral (0 = not neutral, 
1 = neutral) descriptions of the police officer(s) in the videos; the positive description served as 
the reference category. To determine whether the randomization of respondents across the 
varying social context conditions was achieved, we used Ordinary Least Squares (OLS) and 
Logistic regression models to analyze whether there were any statistically-significant differ- 
ences in the basic socio-demographics of the respondents in each grouping. In this exploratory 
exercise, the framing effects dummy variables were used as predictors of age, race, sex, income, 
education, prior crime victimization, and political orientation in the respondents.° Stata 12.0 
statistical software was used to execute all statistical models (StataCorp 2011). 


© OLS regressions were used to analyze the normally-distributed interval-level dependent variables “age” and 
“conservative” scale. Logistic regression models were used to examine the remaining dichotomous dependent 
variables: ““White” (0 = non-white, 1 = white), “male” (0 = female, 1 = male), “income >$40 K” 
(0 = household income <$40,000 per year, 1 = household income >$40,000 per year), “college+” 
(0 = less education than a bachelor’s degree, 1 = bachelor’s degree or higher), and “victim” (0 = household 
member was a prior victim of crime, | = household member was not a prior victim of crime). 
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The results of OLS and Logistic regression models that tested the randomization pro- 
cedure used by KN to allocate respondents among the three framing conditions for the four 
social context issues are available in “Appendix” Table 5. Briefly, there were very few 
statistically-significant (p < .05) differences in the age, race, sex, income, education, prior 
crime victimization, and political orientation in the respondents who were allocated to the 
three framing conditions for the four social context issues. Randomization sometimes 
results in a small number of stochastic differences in group characteristics (Rossi et al. 
2004). This suggests that the manipulated social contextual conditions were successfully 
randomized across the 1,361 respondents in the study.’ 


Videos Shown to Respondents and Outcome Variable 


Respondents viewed three videos, presented in random order.* The videos were chosen to 
vary the nature of the interaction between the police officer and the citizen. However, it is 
important to note here that this study was not designed to identify a representative sample 
of videos drawn from a population of police—citizen interactions so predictions could be 
made about how specific kinds of interactions may modify the impact of contextual 
characteristics on subject perceptions. The research was simply designed to determine 
whether varying social contextual factors affected citizen perceptions of observed police— 
citizen interactions and whether any identified effects were consistent across varied sce- 
narios. In the first video the officers used a higher level of force/conflict than did the 
citizen, who was rude. In the second video, the level of force/conflict was high on both 
sides. In the third video, the level of force/conflict was low on both sides. Videos 2 and 3 
involved interactions between white police officers and African-American citizens. 


Video 1: Rude Citizen/Police Use Force 


This video involved a middle age white male who was pulled over by the police during a 
traffic stop. The citizen was rude toward the police, did not follow police instructions, and 
refused to produce identification. The police officer, a middle-aged white male, responded 
to the belligerent citizen by applying force to restrain and arrest him. 


Video 2: Aggressive Citizen/Police Actively Use Force 


This video showed two white police officers responding to a conflict in a parking lot. The 
citizen, an African-American male, was aggressive in physically pushing the police officers 
who then responded by use of force and ended up trying to handcuff the civilian. 


7 Table 2 identifies only 6 statistically-significant (p < .05) differences across 168 hypothesis tests (3 
videos * 8 contextual dummy variables per video * 7 outcome variables). This is lower than the 8.4 sta- 
tistically-significant results that would be expected by chance alone for 168 hypothesis tests at a .05 level of 
statistical significance. We also ran similar OLS and Probit regression models for all observed variables that 
comprised the latent variables described above; these analyses yielded similar results that supported suc- 
cessful randomization. Further, we created an 81 cell variable representing the different combinations of 
framing conditions and social contextual issues and ran a series of Chi square tests to examine whether the 
observed distributions of cell counts were significantly different from the expected distribution of cells 
counts. We did not find any statistically significant differences between the observed and expected distri- 
butions, suggesting successful randomization by this measure. 


8 These videos showed real-life interactions between police officers and citizens. We acquired the three 
study videos from the internet and edited the footage down to 30s that captured the key events briefly 
described in this section. 
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Video 3: Confused Citizen/Directive Police 


In this video, two white police officers conducted a pedestrian stop of a young African- 
American male. The citizen was polite toward the police and confused about the situation. 
The officers were acquiescent and the citizen complied with police requests. 


Outcome Variable: The Police were Wrong/Should be Punished 


The respondents were asked six questions after viewing each of the three videos. As with the 
other observed variables, these questions were framed as four-point Likert scales (agree 
strongly; agree; disagree; disagree strongly). The questions were: “The police violated the 
law”; “The police engaged in misconduct”; “The police were reasonable and fair” (reverse 
scored); “The officers should be punished” ; “The officers should not patrol the streets” ; and 
“The person should sue the police” (video 1 alpha = .95, video 2 alpha = .95, video 3 
alpha = .94). 


Analytical Strategy 


The data were analyzed using structural equation modeling (SEM) routines. SEMs are 
well-known social science methods that can encompass a broad range of statistical models 
including linear regression, measurement models, and simultaneous equations (Bollen 
1989; Gau 2010). This approach is very attractive to social science analysts due to its 
generality and flexibility. SEMs allow the consideration of simultaneous equations with 
multiple endogenous variables, permit measurement error in exogenous and endogenous 
variables, allow multiple indicators of latent constructs, permit more general measurement 
models than traditional factor-analytic structures, and enable the researcher to specify 
structural relationships among the latent variables (Bollen 1989; Bollen and Long 1993). In 
essence, this flexibility facilitates the comparison of the theoretical model built by the 
analyst to the observed patterns of covariance within the data (Kline 2010). Comparing 
theorized relationships to observed relationships forms the basis for model evaluation and 
for reaching substantive conclusions from SEM analyses. 

Most applications of SEMs are characterized by the following four steps: (1) model 
specification, formulated based on theory and past research in the area; (2) model iden- 
tification, where the researcher determines whether it is possible to find unique values for 
the parameters of the specified model; (3) model estimation, determined by the distribu- 
tional properties of the variables being analyzed; and (4) testing model fit, after estimates 
are obtained the researcher can determine whether the model fits the data (Bollen and Long 
1993). SEM programs translate the user-generated model into a series of covariance 
matrices which is compared with data-generated matrices to determine the extent of the 
match or fit (Kline 2010). The SEM software includes measurement output, which presents 
and assesses the relationships between the endogenous variables and the latent variables, 
and structural output, which presents and assesses the relationships between the exogenous 
independent variables and the dependent variable. A fifth step, model re-specification, is 
added if the model doesn’t fit the data adequately. When the model is re-specified, the four 
primary steps are repeated (sometimes multiple times, see Bollen 1989). After some re- 
specification to ensure that the latent variables (and the endogenous variables that com- 
prised them) and other predictor variables best fit the data, we developed SEMs to analyze 
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the effects of the manipulated social contextual factor variables on respondents’ reactions 
to the three police—citizen interaction videos.’ 

The factor loadings for the endogenous variables that comprised the independent and 
dependent latent variables in the SEMs for the three videos are available in “Appendix” 
Table 6.'° These maximum likelihood estimates represented the measurement portions of 
the SEMs. As can be observed, all factor loadings were consistently high, suggesting very 
strong correlations between the endogenous variables and the underlying latent variables 
measured (Kim and Mueller 1978). 


Results 
Main Effects Models: Manipulated Social Contextual Variables Only 


Table 2 presents the results of the SEMs analyzing the effects of the manipulated social 
contextual variables on the dependent latent variable measuring the extent to which the 
respondents believed the police were wrong and should be punished, excluding the other 
control variables.'' Table 2 also reports the results of three goodness-of-fit metrics: the 
Comparative Fit Index (CFI), the Root Mean Square Error of Approximation (RMSEA), 
and the Standardized Root Mean Residual (SRMR). CFI values larger than .90 and 
RMSEA and SRMR values below .05 suggest very good fit between the hypothesized 
models and the true covariance matrices underlying the data (see Gau 2010).'* Indeed, by 
these standards, our three main effects models fit the observed data very well. 


° We also recognized that our main effects models could be structured as hierarchical linear models to 
analyze multiple observations nested within a single subject. This analysis required restructuring the study 
dataset into a longitudinal panel design where each respondent views three consecutive videos for a total of 
4,083 observations (3 videos * 1,361 respondents). In Stata 12.0, we used the XTSET command to declare 
that the data were structured in the longitudinal panel format and the XTREG command to execute hier- 
archical linear models to analyze the data. Video dummy variables and manipulated social context dummy 
variables were used to measure the main effects of viewing the different police-citizen interaction videos 
and the randomly-allocated framing effects, respectively. Interactive dummy variables were constructed 
(video dummies * manipulated social context dummies) and entered into the model to determine whether the 
observed framing effects were consistent across the three videos. These results were congruent with the 
main effects findings presented here and are available upon request from the authors. 


10 Since they are not directly measured, latent variables do not have intrinsic variances. For the factor 
loadings in Table 3, the variances of the latent variables were constrained to equal “1” for ease of inter- 
pretation (StataCorp 2011, p. 254). These coefficients can be interpreted as correlations that range from “0” 
to “1”. 

'! Unfortunately, small numbers of the 1,361 respondents did not answer all of the study questions for each 
video viewed and for the questionnaire. We handled missing responses through listwise case deletions. As 
Table 2 presents, the number of respondents included in our analyses ranged from 1,232 (90.5 % of 1,361) 
to 1,246 (91.5 %). We used t-tests to examine whether there were any statistically-significant differences 
between included and excluded cases for the covariates used in the SEM analyses presented in Tables 2, 3 
and 4. Our simple comparisons did not reveal any statistically-significant differences between the missing 
cases and the included cases. This suggests that the data were missing at random and, as such, listwise case 
deletions were an appropriate method to address the very modest missing data problem. 


!2 The CFI assesses the fit of a user-specified solution in relation to a more restricted, nested baseline model in 
which the covariances among all input indicators are fixed to zero or no relationship among variables is posited 
(Brown 2006, p. 84). The CFI coefficient value ranges from 0 to 1.00 with values greater than 0.90 indicating a 
reasonably good fit of the hypothesized model (Hu and Bentler 1999). RMSEA takes the error of population 
approximation and degrees of freedom into account and measures the lack of fit of the hypothesized model to the 
population covariance matrix. SRMR is estimated in a similar to RMSEA but does not penalize model com- 
plexity. As a general rule of thumb, SRMR and RMSEA results of 0.05 or less indicates a close approximate fit of 
the model (Hu and Bentler 1999), 
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Table 2 Structural equation models for three police—citizen interaction videos without control variables 


Variables Video | Video 2 Video 3 

b (SE) b (SE) b (SE) 
City neutral —.042 (.032) —.021 (.033) —.028 (.033) 
City negative —.085 (.032)* —.104 (.032)* —.118 (.033)* 
Stop context neutral —.142 (.032)* .024 (.033) —.019 (.033) 
Stop context illegal —.202 (.032)* —.084 (.033)* —.130 (.033)* 
Officer neutral —.048 (.032) —.051 (.033)* .006 (.033) 
Officer negative —.165 (.032)* —.065 (.030)* —.099 (.033)* 
Citizen neutral .042 (.032) —.010 (.033) .068 (.030)* 
Citizen negative .079 (.032)* 019 (.033) .094 (.033)* 
N 1,246 1,238 1,232 
Log likelihood —47,802.689 —48,820.074 —49,232.808 
Likelihood ratio test 
Chi square 1,623.11 1,653.09 1,601.50 
Degrees of freedom 538 538 538 
p > Chi square .000 000 000 
CFI 955 950 951 
RMSEA 043 041 .040 
SRMR 046 O57 056 


For all framing effects dummy variables, positive descriptions served as the reference categories. Dependent 
variable is the “police were wrong/police should be punished” latent variable 


SE standard error 


* p <.05 


As Table 2 reveals, the social contextual framing effects dummy variables were associated 
with statistically-significant impacts on respondent assessments of police actions in all three 
videos. When compared to a positive description of the climate of police-community rela- 
tions in the city, a negative description was associated with a statistically-significant negative 
effect on respondent ratings of police actions (p < .05) in Video 1 (rude citizen, police use 
force), Video 2 (aggressive citizen, police use force), and Video 3 (confused citizen, directive 
police). These relationships suggest that the context of existing police-community relations 
in a given city has a noteworthy impact on how individuals assess the appropriateness of 
police actions across a variety of police—citizen interaction scenarios. 

Relative to a legal police stop descriptions, illegal stop descriptions were associated 
with statistically-significant negative impacts on respondent ratings of police actions 
(p < .05) for all three videos. When respondents viewed Video I (rude citizen, police use 
force), a neutral stop description was also associated with a statistically-significant nega- 
tive effect on respondent assessment of police behavior relative to a legal stop description 
(p < .05). Clearly, the legality of the stops that initiated police—citizen interactions exerted 
a powerful influence on respondent evaluations of police actions in those interactions. 

When compared to a positive description of the police officer’s prior history, a negative 
description was associated with a statistically-significant negative impact on respondent 
ratings of police actions (p < .05) in Video 1 (rude citizen, police use force), Video 2 
(aggressive citizen, police use force), and Video 3 (confused citizen, directive police). 
Conversely, relative to a positive description of the citizen’s prior history, a negative 
description was associated with a statistically-significant positive impact on respondent 
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ratings of police actions (p < .05) in Video 1 and 3. Relative to a positive citizen history 
description, a negative description was not associated with a significant effect on 
respondent ratings of police actions in Video 2 (aggressive citizen, police use force). It is 
important to remember, however, that the citizen in this video immediately attacked the 
responding officers. This obviously criminal behavior probably offset any beneficial effects 
of a positive description of the citizen’s prior history on respondent assessments of police 
actions in the video. 


Main Effects Models: Manipulated Social Contextual Variables with Control Variables 


Table 3 presents the maximum likelihood estimates for the effects of the exogenous latent 
variables and two sets of control variables on the dependent latent variable measuring the extent 
to which the respondents believed the police were wrong and should be punished for Video 1. 
The size, direction, and statistical significance of the coefficients for the manipulated social 
contextual variables were very similar in Table 2. The addition of a series of control variables 
did not substantively change our findings that social contextual frames matter in individual 
assessments of police behavior. The three goodness-of-fit indices reported in both tables 
confirmed that the hypothesized models fit the observed data very well. 

The second column in Table 3 provides the estimates with demographic controls for Video 
1. Table 4 presents the analogous estimates for Videos 2 and 3. Estimates for all three videos are 
quite similar. In Table 3, age, sex, race covariates were not associated with statistically-sig- 
nificant effects on the respondents’ judgments of whether the police officers in the three videos 
were wrong and should be punished. Nonetheless, the non-statistically-significant relationships 
were in the expected direction with older, whites, males'* having more positive assessments of 
police officer actions (Weitzer and Tuch 2006). The insignificant race effect was certainly 
surprising, but was most likely due to the small number of African-Americans in the sample 
(only 14 % of the respondents). However, respondents with bachelor’s degrees or higher levels 
of education were consistently associated with more negative assessments of the actions of the 
police officers (p < .05). Holding other predictors constant, better educated respondents were 
associated with more critical judgments of police actions relative to their less educated coun- 
terparts. Conversely, respondents with incomes over $40 thousand had more positive attitudes 
toward police (p < .05), indicating the complicated effects of socioeconomic status as mea- 
sured by education and income on attitudes toward the police. 

Column 3 reports model estimates with a variety of different control variables entered 
into the model. The effects of the framing variables are little changed. Relative to 
respondents who did not experience a household victimization, respondents from house- 
holds that did experience a prior victimization were associated with more positive ratings 
of police officer actions(p < .05) in Video 2 (aggressive citizen, police use force). How- 
ever, controlling for other covariates, there were not any statistically-significant differences 
in the assessments of police actions made by respondents from households that did and did 
not experience prior victimizations in the other videos. 

Controlling for the other covariates, the latent variables capturing the respondents’ trust 
in government, beliefs on police effectiveness, concerns about neighborhood disorder, and 
fear of crime were not associated with any statistically-significant influences on their 
assessments of police actions in the three videos. Subjects who identified with police 
officers were more likely to appraise their actions more positively. As respondents’ 


'3 The sex effects were statistically significant in Videos 2 and 3 and in the expected direction with males 
expressing more favorable attitudes toward police. 
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Table 3 Structural equation models for police—citizen interaction video 1 


Variables Equation | Equation 2 Equation 3 

b (SE) b (SE) b (SE) 
City neutral —.042 (.032) —.039 (.032) —.036 (.030) 
City negative —.085 (.032)* —.085 (.032)* —.088 (.031)* 
Stop context neutral —.142 (.032)* —.139 (.032)* —.149 (.031)* 
Stop context illegal —.202 (.032)* —.211 (.031)* —.211 (.030)* 
Officer neutral —.048 (.032) —.046 (.032)* —.054 (.031) 
Officer negative —.165 (.032)* —.163 (.031)* —.161 (.029)* 
Citizen neutral 042 (.032) 053 (.032) .044 (.030) 
Citizen negative 079 (.032)* .070 (.032)* .074 (.031)* 
Age >30 .034 (.028) .032 (.027) 
Male 018 (.028) 015 (.028) 
White .039 (.028) .020 (.028) 
College+ —.093 (.028)* —.061 (.028)* 
Income >$40 K 072 (.028)* .049 (.027) 
Prior victim .041 (.028) 
Prior police encounter .026 (.027) 
Prior encounter X PJ LV —.077 (.032)* 
Police identification LV —.229 (.045)* 
Political ideology LV 127 (.048)* 
Trust in government LV —.017 (.048) 
Police effectiveness LV .010 (.051) 
Neighborhood disorder LV —.034 (.042) 
Neighborhood fear LV —.062 (.041) 
N 1,246 1,229 1,229 
Log likelihood —47,802.689 —51,476.01 —54,264.752 
Likelihood ratio test 
Chi square 1,623.11 1,782.695 1,779.41 
Degrees of freedom 538 648 708 
p > Chi square 000 000 000 
CFI 955 950 954 
RMSEA 043 038 035 
SRMR 046 042 026 


For all framing effects dummy variables, positive descriptions served as the reference categories. Dependent 
variable is the “police were wrong/police should be punished” latent variable 


PJ procedural justice, LV latent variable, SE standard error 
* p< .05 


identification with police decreased (moving from “strongly agree” towards “strongly 
disagree”), their assessments of police actions were more negative (p < .05) holding the 
other predictors constant. Subjects who embraced a more conservative political ideology 
and its typically stronger “law and order” orientation viewed police actions in a more 
positive light. Controlling for the other covariates, as respondents described themselves as 
more conservative in their political ideology, their assessments of police actions in all three 
videos became increasingly more positive (p < .05). 
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Table 4 Full structural equation models for three police—citizen interaction videos 


Variables Video 1 Video 2 Video 3 

b (SE) b (SE) b (SE) 
Age >30 .032 (.027) .039 (.026) .003 (.027) 
Male O15 (.028) 159 (.026)* .076 (.028)* 
White 020 (.028) 030 (.028) .028 (.029) 
College+ —.061 (.028)* —.083 (.027)* —.068 (.028)* 
Income >$40 K .049 (.027) .047 (.027) .032 (.028) 
Prior victim 041 (.028) .067 (.028)* .049 (.029) 
Prior police encounter 026 (.027) .019 (.027) —.017 (.028) 
Prior encounter X PJ LV —.077 (.032)* —.035 (.032) —.091 (.033)* 
Police identification LV —.229 (.045)* —.213 (.045)* —.206 (.046)* 
Political ideology LV 127 (.048)* 104 (.048)* 180 (.049)* 
Trust in government LV —.017 (.048) .062 (.048) —.008 (.050) 
Police effectiveness LV .010 (.051) —.037 (.051) .094 (.052) 
Neighborhood disorder LV —.034 (.042) —.042 (.036) .076 (.041) 
Neighborhood fear LV —.062 (.041) —.054 (.047) -059 (.040) 
City neutral —.036 (.030) —.006 (.030) —.024 (.031) 
City negative —.088 (.031)* —.072 (.030)* —.123 (.030)* 
Stop context neutral —.149 (.031)* 035 (.031) —.037 (.031) 
Stop context illegal —.211 (.030)* —.085 (.030)* —.129 (.030)* 
Officer neutral —.054 (.031) —.066 (.030)* 005 (.031) 
Officer negative —.161 (.029)* —.068 (.030)* —.099 (.031)* 
Citizen neutral 044 (.030) 014 (.030) O71 (.030)* 
Citizen negative .074 (.031)* 038 (.030) 087 (.030)* 
N 1,229 1,240 1,232 
Log likelihood —54,264.752 —51,914.123 —55,422.515 
Likelihood ratio test 
Chi square 1,779.41 1,712.49 1,941.26 
Degrees of freedom 708 708 708 
p > Chi square 000 000 .000 
CFI 954 949 .946 
RMSEA 035 038 038 
SRMR 026 026 .027 


For all framing effects dummy variables, positive descriptions served as the reference categories. Dependent 
variable is the “police were wrong/police should be punished” latent variable 


PJ procedural justice, LV latent variable, SE standard error 
* p< .05 


Holding the other variables constant, having a prior encounter with the police or not was 
not associated with any statistically-significant differences in respondent assessments of 
police actions. However, for those who did have a prior encounter, the nature of their 
experiences with the police in their most recent interaction associated with a statistically- 
significant effect (p < .05) on their assessments of police actions. As these respondents 
perceived their prior interaction with the police to be increasingly less procedurally just 
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(moving from “strongly agree” towards “strongly disagree”), their assessments of police 
behavior became increasingly negative. 

An important aspect of column 2 Table 3 is that it allows us to compare the size of our 
social context variables with the effect of different demographic variables. Specifically, we 
focus on education because its effect is the largest and its substantive importance. An 
individual’s level of educational attainment is regarded as a very important predictor of a 
variety of life outcomes and behaviors such as social status (Beeghley 2004), voting 
(Wolfinger and Rosenstone 1980), and criminal activity (Gottfredson 1981). Indeed, in this 
analysis, the only control variable that produced a consistent effect across all three videos 
was the level of education of the respondent. In Table 3, column 2, the reported effect size 
for having some or more college is —.093 with a standard error of .028.'* 

The manipulated social contextual variables yield effect sizes that are similar, and 
sometimes larger in size than the effects of educational attainment on individual appraisals 
of police behavior. Describing the city in negative as opposed to positive terms prior to 
seeing a video has a —.085 effect on a respondent’s evaluation of police behavior. 
Describing the circumstances of the stop in the video in either neutral (—.139) or as illegal 
(—.211) have larger negative effects than that of education. A negative descriptive of the 
police officer also leads to a much larger effect (—.163). Describing the citizen in negative 
terms leads to a somewhat smaller effect (.070) than education. 


Conclusion 


The results of our randomized factorial experiment suggest that the immediate priors a 
person has about a situation can have a considerable effect on their evaluation of that 
situation, with effects as large as or even larger than a key demographic variable such as 
education. As such, what these results show is that the perspective a person brings to 
evaluating a situation can have a substantial effect independent of what actually occurred in 
that situation. This is an important insight in considering how citizens are likely to evaluate 
police behavior. Even in situations where police do act properly, citizens, because of their 
prior perspectives, may well evaluate police as not having done so. In many ways, our 
findings are consistent with research on the reliability of eyewitness testimony that suggests 
individual memories and perceptions can be manipulated, altered, and biased (Loftus et al. 
2008). Beyond the objective events observed, eyewitness accounts can be notably influ- 
enced by prior attitudes and values held by individual observers (Wells and Olson 2003). 

Consistent with cognitive and social psychology research on priming effects, social 
contextual factors shape citizen evaluations of police wrongdoing even when people see the 
same event. Depending on the content of the social context primes, people can react differ- 
ently to an identical stimulus (Fiske and Taylor 1991; Balcetis and Lassiter 2010). 


'4 Arguably the effect of Education should be estimated without controlling for income, since presumably 
part of the effect of Education is mediated through Income. Dropping Income as a control only results in the 
effect of Education being reduced—with the effect dropping to —0.78 (SE = .028, p < .05). In our sample, 
levels of Education are very modestly correlated with levels of Income (r = .28). In reduced regression 
models of the effects of Education and Income on respondent assessment of police behavior, we find that 
both are statistically significant predictors of the outcome variable in all three videos. Controlling for 
Income, increasing levels of Education are associated with more negative perceptions of observed police 
behavior. This suggests that more educated people are more critical appraisers of police behavior inde- 
pendent of how much money they earn. Clearly, future research in this area should seek to better disentangle 
these complicated effects. 
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Psychological research on consumer preferences suggests that priming a particular attribute 
increases the likelihood that this attribute will be used to interpret product information in an 
advertisement, and thus influences the evaluation of the advertised brand (Gardner 1983; Herr 
1989). For the police, the activation of positive or negative attributions of observed police 
actions can stimulate or hinder citizen cooperation in resolving particular incidents or 
assistance in control of crime and disorder problems. Coupled with the existing research on 
procedural justice and police-community relations (Mazerolle et al. 2013; Reisig et al. 2007; 
Tyler 2003), the implications of these findings for the delivery of police services may be 
profound. While concentrated social disadvantage, racism, and isolation from political power 
are underlying causes, urban riots are usually sparked by negative perceptions of observed 
police behavior in a climate of persistently poor police-community relations (Skolnick and 
Fyfe 1993; U.S. National Advisory Commission on Civil Disorder 1968). 

Citizen perceptions of their interactions with the police are strongly shaped by pre- 
existing opinions of the police (Brandl et al. 1994; Rosenbaum et al. 2005). Vicarious 
experiences also influence the way citizens interpret and evaluate their own encounters with 
the police (Brunson 2007; Gallagher et al. 2001; Hohl et al. 2010; Weitzer and Tuch 2006). 
These global, pre-existing views of the police influence citizen assessments of police 
behavior in specific interactions and also their general views of the police. Our research adds 
to this line of scientific inquiry by demonstrating that situation-specific perceptions held just 
prior to viewing police—citizen interactions have strong impacts on citizen perceptions of the 
rightfulness or wrongfulness of police behavior during those interactions. Broader social 
contextual factors, such as the climate of police-community relations in a particular city, and 
immediate contextual factors, such as the lawfulness of the stop and the work history of the 
officer making the stop, clearly matter in citizen appraisals of police behavior. For instance, 
an older, upper-middle-class, conservative white male may hold innately positive views of 
the police and, in general, he may be very willing to cooperate with police. Our findings 
suggest that his assessment of an observed police action would be negatively impacted by 
immediate a priori knowledge that the police department in his city is being investigated for 
persistent patterns of abusive behavior, the officer in the encounter is being investigated on 
corruption charges, and/or the stop that initiated the encounter was illegal. This an important 
refinement to the existing literature that has generally examined more static measures of pre- 
existing views on the legitimacy of police behavior. 

The results of our randomized factorial experiment provides some support to the perspective 
that police executives and organizations can positively influence citizen interpretations of police 
actions by striving to create a climate of positive police-community relationships in cities 
(Skogan and Frydl 2004; Skolnick and Fyfe 1993). In essence, the police can shape the social 
contexts that influence citizen perceptions of police activities on the street. Community policing 
programs, in particular, seem to be well positioned to improve citizen judgments of the appro- 
priateness of police behavior in specific incidents. The available evaluation evidence suggests that 
community policing programs enhance police-community relationships and improve citizens’ 
judgments of police actions (Skogan and Frydl 2004; Weisburd and Eck 2004). Our research 
suggests that these positive impressions of police departments and individual officers transfer to 
the interpretation of the rightfulness or wrongfulness of particular police actions made in specific 
interactions with citizens. As such, the police may be able to create a “reservoir of goodwill” with 
the community that can extend to citizen interpretations of enforcement work on the street that 
could negatively impact the willingness of community members to cooperate with the police. 

Equally important, police executives and organizations need to ensure that officers are 
lawful in their interrogations, searches and seizures, and use of force (Fyfe 2002; Gould and 
Mastrofski 2001). Police lawlessness seriously undermines citizen confidence and trust in 
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law enforcement agencies (Skogan and Meares 2004). Citizens expect the police to enforce 
laws fairly according to the laws and rules that circumscribe their enforcement powers. In 
this study, relative to positive conditions, respondents made negative appraisals of police 
actions in contexts where police departments were suggested to be known for civil rights 
violations and other unlawful actions, particular officers were suggested to be problematic 
in their police work, and the stops that initiated the police—citizen interaction were illegal. 

Our research, like prior studies, suggests that public judgments about the legitimacy of the 
police are influenced by their assessments of the manner in which the police exercise their 
authority (Tyler 2004, 2006). Under experimental conditions, the procedural justice experi- 
enced by respondents in their most recent encounter with the police impacted their assess- 
ments of police actions in all three videos independent of the social contextual frames applied. 
Past experiences with police officers shaped citizen attributions of the legitimacy of those 
actors and the institutions they represent. When citizens have prior interactions where they 
believe the police treated them disrespectfully and unfairly, they negatively assess the actions 
of police officers in an independent subsequent event. Clearly, to improve citizen judgments 
of handling of specific incidents, police departments need to ensure that its officers are being 
procedurally just in their interactions with the public. As this research and prior research 
seems to indicate even if there is good relations between the community and their police 
force, a single negative encounter could overcome all the good faith effects (Brunson 2007; 
Gallagher et al. 2001; Hohl et al. 2010; Warren 2011; Weitzer and Tuch 2006). 

This study, like most social science inquiries, has several limitations that future 
investigations could improve upon. Most importantly, there are very few young, non-white 
males in our study. It would be very important to determine the salience of social con- 
textual factors for a population that is generally subjected to higher levels of police 
scrutiny and hold much more negative perceptions of the police (Brunson and Weitzer 
2009; Weitzer and Tuch 2006). It is also important to remember the artificial context of the 
research study. While the use of videos of actual encounters is a significant improvement 
over written vignettes, respondents may have responded differently if they were physically 
present at the scene of the actual police—citizen interactions. Future scientific inquiries 
should also consider whether contextual factors shape procedural justice judgments of 
observed police—citizen interactions rather than limiting their assessments to perceptions of 
whether the police were wrong and should be punished. Indeed, there are a variety of 
refinements that future studies could make, such as including measures of global attitudes 
toward the police rather than limiting questions to police officers in the respondents’ 
neighborhoods only and examining the role of violent household victimization rather than 
any household criminal victimization. 

While complex and challenging, police executives and police organizations need to create 
positive social contextual environments in their jurisdictions by ensuring that “good polic- 
ing” is uniformly applied to all neighborhoods. The ideas of legitimacy and procedural 
justice are central to this perspective. Police departments need to engage community policing 
principles, ensure that officers are lawful in their enforcement actions, and all citizens are 
treated in a fair and respectful manner. In return, police departments will accrue a variety of 
benefits such as improving their ability to control crime and hold offenders accountable and 
receiving more favorable interpretations of their enforcement actions on the street. 


Appendix 


See Tables 5 and 6. 


va Springer 


J Quant Criminol 


(PLD) 617 
a(€L'T) SBE 
(ZL'1) 66'1 
(PLD) LVI- 
(PL'T) 60°I- 


«(PL 1) 6b S— 
(IL'D) €67— 
(€L'T) OFT 
(LD Or 
(CLT) 00'I- 
(ELT) 10°%— 
(LD) 0@I- 
(ZL'1) 60° 


(PLT) 80°1 
(ZL) 18°— 
(ELD) St7— 
(LY) OL I- 
(€L'L) 80° 
(ZL'1) v6" 
(ILD) €77 
(PLD) SLiI- 


x(1V'€) 99°87 


STO 
OATVAIOSUOD, 


(€1) pO — 
(€1") €0° 
(€1) LO 
(pI) ZO'— 
(€1') 90°— 


(€1) 17 
(€I') 80° 
(€l) 
(€I') $0” 
(€1') So'— 
(€r') 81— 
(€1") LO” 
(€1) 10'— 


(€1") ZO" 
(€l) ST 
(el) LV- 
(€1) 81— 
(€1) vl 
(EL) 12 
(€1) 00° 
(pl) LV- 


x(TIT) 660° 


ONSISOT 
WNSTA 


(pI) r0'— x(91') 6€°— 
(FI) 60° (91°) 0@— 
(PL) CZ (91°) 90° 
(pl) LV (LV) 90°— 
(rr) LV- (LI) 60°— 
(PL) TP (LV) 10- 
(€1) 10"— (91°) pl — 
(pI) €1 (91°) ST'— 
(rl) eV (91°) 0" 
(v1) 80°— (91°) So'— 
(rI") SO” (91) IT 
w(€T) 9€° (91°) €0° 
(v1) ZO"- (91°) LV 
(hI) 70° (LI) €7 
(v1) €0°— (91°) pr 
(Pl) OV'— (LV) €V 
(er) €1°— (91°) 61'— 
(y1) LO’ (91°) 10° 
(v1) 60°— (LT) 6 
(v1) LO’ (91°) I0— 
(v1) 90°— (L1°) 60°— 
x(6VT') TS x(O€E") TST 
oSISO'T O1SISO'T 
+o89110D M Op$< sutoouy 


(€1) 07 — 
(€1’) 61°— 
(€1) 80°— 
(pI) ZO" 

(VL) LV 


(€1) 10° 
(€1') SI'— 
(€I') 60°— 
(€1") 80° 
(€1') 80° 
(€1') 10" 
(€I") €0° 
(€r) LV 


(rr) IV— 
(€1) 60°— 
(EL) 10° 
(€1) vl 
(er) 1 
(EI) 10° 
(€l) €1— 
(€1') ZO"— 


x(VET) P8P" 


ONSIBO'T 
eI 


(ST) IV 
(ST) 60° 
($1) LO 
(S1°) 80° — 
(ST") LO'— 


($1) 90°— 
(ST) ZO” 
(ST) LV 
(ST) 61° 
(ST) 70" 
(SI) ZO'— 
(ST) Ol'— 
(ST) €0" 


($1) IV— 
(ST) 80° 
(SI) po- 
(St) vl — 
(SI) ST- 
($1) 00° 
(ST) 60" 
($1) Ol'— 


x(S67') 118° 


O1SISO'T 
OU. 


(€O'1) OF 
(10°) So” 
(ZO'1) TET 
(€O'1) EL’— 
(€O'L) Er 


(€O'1) LY — 
(10'D) 6€°— 
(ZO'L) Sr 
(0D) vr 
(ZO'L) ELT 
(€O'L) 76° 
(ZO'L) So’— 
(ZOD) €7— 


(ZOD) 97 — 
(ZOD) 6r'— 
(ZOD) 7S 
(10°D) 6¢°— 
(€O"D) 65° — 
(ZOD) €L’— 
(ZOD) L8" 
(€0°D) SL'I- 


x(ZO'T) STL 


STO 
osV 


[eyNeu I99qjJO 
[eso] Ixa}U09 dois 
Teynou }xo}U09 doy 
oanesou AyD 
Teynou AyD 

€ APIA 

OANLSOU UIZTID 
[erneu waZnIO 
OANLSIU JOOYJO 
Teneu I20yjJO 
[eso] 1xa}U09 dois 
Tennou jxaU09 dois 
aanesou AyD 
yemneu AqID 

f 9APIA 

aanesou UaZtID 
Teneu waz 
DANVSOU IOOYJO 
[Tenneu I20yJO 
[esa] IxaU09 dois 
[enou xa}U09 dois 
aanesou AyD 
yemneu AqID 

T OaplA 

ydao10)Uy 


[PPO 
aqqeite A 


LOC] = N ‘Sonsiojovreyo quopuodsai P9}D9][9S IOF SJIaffo SUTUIRIT JO UOTBZIWOpUeI 1S9} OF SUOISSOISOI O1}SISO] pue STO § IqeL 


pringer 


NS 


J Quant Criminol 


(PLL) L8'7 
(LD 67 — 
(ZL) So'— 


STO 
SANRAIASUOD) 


(€1') 90° 
(€L) Lo'— 
(€L) vl 


UIA 


(v1) ZO"— 
(v1) 60° 
(v1) r0"— 


dSISO'T 
+a3a][09 


(91°) Ol'— 
(91°) SO” 
(LI) ZO"— 


IISISO'T 
M Or$< stutoouyT 


(rT) €0° 

(el) er 

(ET) 60° 
oNSISO'T 
sR 


x(S1") €€° 
x(ST") LE 
(ST) 61” 


OSISO'T 
SHUM 


(€0'L) €7 
(ZOD) 89'1 
(ZO'L) 9€'1 


STO 
osV 


co > dx 


aANesou EZ 
[enneu uaznio 
SANRZOU IBDYJO 


oNSISO'T 


TPO 
ayqeire A, 


ponunuos ¢ 3quy 


pringer 


vans 


J Quant Criminol 


Table 6 Factor loadings for endogenous variables comprising latent variables in SEMs 


Latent variables Video 1 Video 2 Video 3 
b (SE) b (SE) b (SE) 


Identification w/police 


Share similar views .716 (.016)* .724 (.016)* -718 (.016)* 
Understand police actions .722 (.017)* .708 (.017)* -707 (.018)* 
Like police in my area .784 (.014)* .786 (.014)* -781 (.014)* 
Respect my values -701 (.018)* .705 (.018)* -701 (.018)* 
Value my contributions -715 (.017)* .719 (.017)* -722 (.017)* 
Political ideology 

Liberal versus conservative 891 (.009)* .890 (.009)* -889 (.009)* 
Democrat versus Republican -722 (.015)* 727 (.015)* -723 (.015)* 
Liberal scale (reverse) 828 (.011)* 827 (.011)* -831 (.011)* 
Conservative scale TAT (.014)* .749 (.014)* -746 (.014)* 
Trust in government 

Improve living standards 842 (.012)* 841 (.012)* -839 (.012)* 
Gov’t doing too much .847 (.012)* .842 (.012)* -841 (.012)* 
Should help minorities 711 (.017)* .720 (.017)* 714 (.017)* 
Police effectiveness 

Reduce crime in my area .785 (.014)* .789 (.014)* -786 (.014)* 
Quick response to calls .769 (.014)* .767 (.014)* -766 (.014)* 
Help people when asked 860 (.011)* 856 (.011)* -859 (.011)* 
Neighborhood disorder concerns 

Beer bottles on street .689 (.017)* .681 (.017)* -689 (.017)* 
Graffiti on walls .695 (.016)* .695 (.017)* -700 (.017)* 
Gangs on the streets .815 (.012)* .816 (.012)* -815 (.012)* 
Public drinking .803 (.012)* .800 (.012)* -801 (.012)* 
Drug selling 793 (.013)* .786 (.013)* -795 (.013)* 
Neighborhood fear of crime 

Fear of victimization 745 (.017)* .743 (.017)* -743 (.017)* 
Home burglarized 861 (.015)* 858 (.015)* -858 (.015)* 
Street violence victim .620 (.021)* .621 (.021)* -613 (.022)* 
Procedural justice w/police 

Described my situation .758 (.016)* .746 (.017)* -749 (.017)* 
Treated like others .732 (.018)* .732 (.018)* -731 (.018)* 
Police decision based on facts 839 (.012)* .840 (.012)* -839 (.012)* 
Police cared about concerns 853 (.011)* .857 (.011)* 854 (.011)* 
Treated me politely 895 (.009)* .889 (.009)* 894 (.009)* 
Respected my rights 898 (.009)* .892 (.009)* -899 (.009)* 
Police wrong/should punish 

Police violated law .882 (.006)* .868 (.007)* -842 (.006)* 
Police engaged misconduct 919 (.005)* .907 (.005)* 912 (.006)* 
Police were fair (reverse) .797 (.011)* .767 (.012)* -707 (.015)* 
Police should be punished .964 (.003)* .950 (.004)* 942 (.004)* 
Police should not work street 914 (.005)* .916 (.005)* -902 (.005)* 
Police should be sued .780 (.011)* .821 (.010)* 849 (.006)* 
* p< .05 
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